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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH{S) OR THIRTY (30) DAYS. 
WHICHEVER IS LONGER. FROM THE MAILING DATE OF THIS COMMUNICATION. 

• Extensions of time may be available under the provisbns of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period yN\\) apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three nwnths after t he mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)81 Responsive to communiGation(s) filed on 30 June 2003 . 
2a)n This action is FINAL. 2b)l3 This action is non-final 

3) 0 Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 
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?)□ Claim(s) is/are objected to. 
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Application Papers 
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Applicant may not request that any objection to the drawlng(s) be held in abeyance. See 37 CFR 185(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
11 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or fonn PTO-152. 

Priority under 35 U.S.C. § 119 
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DETAILED ACTION 



Claim Rejections - 35 USC § 102 



1 . The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 



2. Claims 1-4, 6, 8-11, 13, 15-18, and 20 are rejected under 35 U.S.C. 102(e) as 
being anticipated over Wang et al. (US PGPub 2002/0165719). 

As to claim 1, Wang discloses a computer interface system (see 
paragraphs [0027] lines 1-6), comprising: a microphone that receives audio input 
from a user (see paragraph [0008] lines 3-5); a voice recognition mechanism 
(see paragraphs [0018]); and a graphical user interface that prompts the user for 
expected inputs that the user can speak at designated points in a dialog (see 
paragraph [0010] lines 3-7) according to a specified grammar (see paragraphs 
[0008] lines 7-9); wherein prompts may specify the type of expected input (drop 
down boxes are used in the GUI to Indicate expected values, see figure 1); 
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wherein prompts may specify words that are recognized by the system (see 
paragraph [0008] lines 7-9). 



As to claim 6, Wang discloses a computer program product in a computer 
readable medium for use in a computer interface system (see paragraph [0027]), 
the computer program product comprising: first instructions for receiving audio 
input from a user (see paragraph [0008] lines 3-5); second instructions for 
automatic voice recognition (see paragraphs [0018]); and third instructions for 
displaying a graphical user interface that prompts the user for expected inputs 
that the user can speak at designated points in a dialog (see paragraph [0010] 
lines 3-7) according to a specified grammar (see paragraphs [0008] lines 7-9); 
wherein prompts may specify the type of expected input (drop down boxes are 
used in the GUI to indicate expected values, see figure 1 ); wherein prompts may 
specify words that are recognized bye the system (see paragraph [0008] lines 7- 
9). 



As to claim 15, Wang discloses a method for interfacing between a 
computer and a human user (see abstract line 2), the method comprising the 
computer-Implemented steps of: receiving audio input from the user (see 
paragraph [0008] lines 3-5); interpreting the audio input via voice recognition (see 
paragraphs [0018]); and displaying a graphical user interface that prompts the 
user for expected inputs that the user can speak at designated points in a dialog 
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(see paragraph [0010] lines 3-7) according to a specified grammar (see 
paragraphs [0008] lines 7-9); wherein prompts may specify the type of expected 
input (drop down boxes are used in the GUI to indicate expected values, see 
figure 1); wherein prompts may specify words that are recognized bye the system 
(see paragraph [0008] lines 7-9). 

As to claims 2, 9, and 16, Wang discloses a multi-modal input system 
wherein prompts that represent non-terminal tokens in the grammar are replaced 
with one of a set of other prompts in the grammar in response to the spoken 
input (see paragraph [0082] lines 1-7). 

As to claims 4, 1 1 , and 18, Wang discloses said system with at least one 
speaker that provides audio prompts for expected inputs (see paragraph [0064] 
lines 7-11). 

As to claims 6, 13, and 20, Wang discloses said system with a graphical 
user interface that further comprises a pull-down menu (see figure 1). 
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Claim Rejections ■ 35 USC § 103 
3. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary sl<ill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 



4. Claims 5, 7, 12, 14, 19, and 21 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Wang et al. (US PGPub 2002/0165719) in view of Katsuranis (US 
PGPub 2005/0021336). 

As to claims 5, 12, and 19, Wang does not specifically disclose a 
multi-modal input system that the prompt may further comprise a second 
graphical user interface window. Katsuranis teaches a multi-modal input 
system that displays and controls the content of a second graphical 
application window while in a first graphical application window in a 
windowed computing environment having a voice recognition engine (see 
abstract lines 1-5). It would have been obvious to one having ordinary 
skill in the art at the time the invention was made to have modified the 
multi-modal input system of Wang with the feature of the first and second 
graphical user interface window as taught by Katsuranis. Doing so would 
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have allowed a user using the system to be able to organize the graphical 
window he or she is working with and eliminate the fmstration of having to 
toggle through numerous windows just to refer from one window to the 
other (see paragraphs [0004], [0005], [0033]). 

As to claims 7, 14, and 21, Wang does not specifically disclose a 
multi-modal input system that comprises a set of reserved words that 
activate specified prompts when spoken by the user. Katsuranis teaches 
using key word commands to open, view, retrieve, and much more (see 
abstract lines 5-10). It would have been obvious to one having ordinary 
skill in the art at the time the invention was made to have modified the 
multi-modal input system of Wang with the features of a set of key word 
commands as taught by Katsuranis. Doing so gives the system key words 
that are not necessarily a non-terminal command but are intended to 
facilitate the navigation and experience of the user. 



5. ^ Claims 5, 7, 12, 14, 19, and 21 are rejected under 36 U.S.C. 103(a) as being 
unpatentable over Wang et al. (US PGPub 2002/0165719) in view of Katsuranis (US 
PGPub 2005/0021336) as applied to claims 1 , 8, and 15 and in further view of Dantzig 
et al. (US PGPub 2003/0071833). 
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As to claims 3, 10, and 17, Wang discloses a multi-modal input 
system wherein the graphical user Interface is built automatically from a 
single grammar specification (image grammars can be used to have visual 
representation of voice recognition thus making it automatic, see 
paragraph [0064] lines 1-6). 

Wang or Katsuranis do not disclose using a dictionary for the creation of 
the graphical user interface. Dantzig teaches creating a multimodal 
interface with GUI and speech recognition (see abstract). In the GUI a 
dictionary Is use for construction of the interface (see paragraph [0092] 
lines 1-5). It would have been obvious to one having ordinary skill in the 
art at the time the invention was made to have modified the multi-modal 
input system of Wang and Katsurani with the features of a dictionary as 
taught by Dantzig. Doing so would allow for addition of new grammars 
with out have to constantly add new meanings to the database of the 
system. 
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Conclusion 

A note has been made to notify ttie appropriate parties that the examiner 
has moved from Art Unit 2609 to 2626. 

Any inquiry concerning this communication should be directed to Josiah 
Hernandez whose telephone number is 571-270-1646. The examiner can 
normally be reached from 7:30 pm to 5:00 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, David Hudspeth can be reached on (571) 272-7843. The 
fax phone number for the organization where this application or proceeding is 
assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from 
the Patent Application Information Retrieval (PAIR) system. Status information 
for published applications may be obtained from either Private PAIR or Public 
PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see http://pair- 
direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll- 
free). 
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